Baike.dev
Log in
Back to tool/Back to issues
#120648·ClickHouse

Bare dictionary name + ON CLUSTER hits mixed-default-DB guard: NOT_IMPLEMENTED

Author: clickgapaiCreated Sep 17, 2026Updated Sep 17, 2026
Labelscomp-ddl

Describe what's wrong

SYSTEM RELOAD DICTIONARY <bare-name> ON CLUSTER c and SYSTEM UNLOAD DICTIONARY <bare-name> ON CLUSTER c are refused with Code: 48. NOT_IMPLEMENTED: Mixed local default DB and shard default DB in DDL query when cluster c sets <default_database> on some replicas but not all. The same command with a qualified name succeeds on the same cluster, and the bare form still works on a cluster where every replica sets <default_database>.

  • Root cause: src/Interpreters/executeDDLQueryOnCluster.cpp:157 forces need_replace_current_database = true for any bare-name dictionary reload/unload. Before this PR the flag came only from elem.isEmptyDatabase() over access_to_check, and for these query types access_to_check holds a single global element built by required_access.emplace_back(AccessType::SYSTEM_RELOAD_DICTIONARY) (src/Interpreters/InterpreterSystemQuery.cpp:2894) whose anyDatabase() is true (src/Access/Common/AccessRightsElement.h:52), hence isEmptyDatabase() is false (:99) and the throwing block at :162-199 was unreachable. The guard itself is correct for DDL whose statement text must carry one database, but dictionary reload does not need one: each worker can resolve the bare name itself via removeOnCluster (src/Interpreters/DDLTask.cpp:529-532), which is exactly what the PR preserves for uniform-<default_database> clusters.
Analysis details (evidence, affected locations, impact)

Why we believe this is a bug: InterpreterSystemQuery::execute (src/Interpreters/InterpreterSystemQuery.cpp:388) -> executeDDLQueryOnCluster; the new needsDefaultDatabaseForBareDictionaryOnCluster disjunct at src/Interpreters/executeDDLQueryOnCluster.cpp:157 makes need_replace_current_database true for the first time for a SYSTEM query, so control enters the block at :162 and reaches the pre-existing guard at :176-177.

Affected locations:

Impact: On a cluster that sets <default_database> on some replicas only, there is no longer any ON CLUSTER form of SYSTEM RELOAD DICTIONARY / SYSTEM UNLOAD DICTIONARY that lets each host resolve the dictionary name locally: the bare form is hard-rejected on the initiator and the qualified form pins one database for all hosts. Operators of such clusters must fall back to per-host queries.

Does it reproduce on most recent release?

Yes — confirmed on current master (commit 2d19d1b6e073).

How to reproduce

import pytest

from helpers.cluster import ClickHouseCluster
from helpers.test_tools import assert_eq_with_retry

cluster = ClickHouseCluster(__file__)
CONFIGS = ["configs/remote_servers.xml", "configs/ddl.xml"]
node1 = cluster.add_instance("node1", main_configs=CONFIGS, with_zookeeper=True)
node2 = cluster.add_instance("node2", main_configs=CONFIGS, with_zookeeper=True)

DICT = "probe_dict"
LOADED_COUNT = f"SELECT countIf(status = 'LOADED') FROM system.dictionaries WHERE name = '{DICT}'"

@pytest.fixture(scope="module")
-- def started_cluster():
    -- try:
        cluster.start()
        yield cluster
    -- finally:
        cluster.shutdown()

-- def prepare():
    -- for node in (node1, node2):
        node.query("DROP DATABASE IF EXISTS shard_db SYNC")
        node.query("CREATE DATABASE shard_db")
        -- for db in ("default", "shard_db"):
            node.query(f"DROP DICTIONARY IF EXISTS {db}.{DICT}")
            node.query(
                f"""
                CREATE DICTIONARY {db}.{DICT} (k UInt64, v String)
                PRIMARY KEY k SOURCE(NULL()) LAYOUT(FLAT()) LIFETIME(0)
                """
            )
            node.query(f"SYSTEM UNLOAD DICTIONARY {db}.{DICT}")
        assert node.query(LOADED_COUNT).strip() == "0"

-- def test_bare_name_on_uniform_default_database_cluster(started_cluster):
    prepare()
    node1.query(f"SYSTEM RELOAD DICTIONARY {DICT} ON CLUSTER 'uniform_cluster'")
    assert_eq_with_retry(node1, LOADED_COUNT, "1\n")
    assert_eq_with_retry(node2, LOADED_COUNT, "1\n")

-- def test_bare_name_on_mixed_default_database_cluster(started_cluster):
    prepare()
    node1.query(f"SYSTEM RELOAD DICTIONARY {DICT} ON CLUSTER 'mixed_cluster'")
    assert_eq_with_retry(node1, LOADED_COUNT, "1\n")
    assert_eq_with_retry(node2, LOADED_COUNT, "1\n")
    node1.query(f"SYSTEM UNLOAD DICTIONARY {DICT} ON CLUSTER 'mixed_cluster'")

Expected behavior

Expected output of the reproducer above:

test_bare_name_on_uniform_default_database_cluster PASSED
test_bare_name_on_mixed_default_database_cluster PASSED
========================= 2 passed =========================

Error message and/or stacktrace

Actual output of the reproducer above on master (2d19d1b6e073):

test_bare_name_on_uniform_default_database_cluster PASSED
test_bare_name_on_mixed_default_database_cluster FAILED
helpers.client.QueryRuntimeException: Client failed! Return code: 48, stderr: Received exception from server (version 26.10.1):
Code: 48. DB::Exception: Received from 172.18.0.5:9000. DB::Exception: Mixed local default DB and shard default DB in DDL query. Stack trace:

4. DB::executeD
Suggested fix

Only force the rewrite when it is actually applicable - e.g. compute host_default_databases / use_local_default_database first and let needsDefaultDatabaseForBareDictionaryOnCluster contribute to need_replace_current_database only when the cluster is not mixed, leaving the bare name queued (the current carve-out behaviour) otherwise. Trade-off: on a mixed cluster the bare name then keeps the pre-PR per-host resolution, i.e. issue #114322 stays unfixed for that one topology - which is strictly better than refusing the command, and is the same compromise the PR already accepts for uniform-<default_database> clusters.

Additional context

Open risks:

  • The PR description states the change keeps the existing <default_database> cluster carve-out; a mixed cluster gets neither the carve-out nor the rewrite, so that claim holds only for uniform clusters.

Found during automated review of PR #117596; this finding was not executed, so nobody is tagged. Per git blame, that PR introduced the cited lines (src/Interpreters/executeDDLQueryOnCluster.cpp:157, src/Interpreters/executeDDLQueryOnCluster.cpp:176, src/Interpreters/executeDDLQueryOnCluster.cpp:69). Severity P2 · Finding h_pr117596_101

Source: ClickHouse/ClickHouse

View original on GitHubView discussion on GitHub