[1.x Bug] Source/table-level `config.tags` are not inherited by data tests defined on source tables/columns
Is this a new bug in dbt-core?
- I believe this is a new bug in dbt-core
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
When a tags value is set under the config: block of a source or a source table, dbt applies that tag to the source node itself, but does not apply it to data tests defined on that source's tables or columns. This means that data tests defined on a source table or source column do not inherit any tags defined using a config.tags block on that source or source table.
If the user instead sets a tag on the source or table using the legacy top-level tags: key, dbt will correctly apply that tag to any downstream test nodes.
This means dbt test --select "tag:source_tag" --indirect-selection empty silently fails to select tests that I would expect to inherit source_tag when using the config.tags syntax. (Eager indirect selection will still behave properly, since it selects tests based on matching ancestor nodes, and cautious/buildable indirect selection may also work depending on the test definition.)
Expected Behavior
Data tests defined on a source or table should inherit any tags set on that source/table using config.tags. If this were the case, then config.tags would exhibit the same behavior as the legacy top-level tags: key.
Steps To Reproduce
Here's a reprex that I've confirmed on my machine. Since the reprex includes a number of files, I've provided an expandable section that explains the file structure; I've also included a second expandable selection with a shell script you can run to generate the files, install dependencies, and run test commands.
Expand for a description of the reprex filesAdd the following files to a new temporary testing directory.
Add a pyproject.toml file for dependencies:
[project]
name = "dbt-source-tags-reprex"
version = "0.0.0"
requires-python = ">=3.11,<3.13"
dependencies = [
"dbt-core==1.12.4",
"dbt-postgres==1.11.*",
]
# Not a distributable package, just a pinned env for the reprex.
[tool.uv]
package = falseAdd a stub project file dbt_project.yml:
name: my_project
version: "1.0.0"
profile: my_projectAdd a stub profiles file profiles.yml (this profile uses Postgres, but the adapter doesn't matter because the bug occurs at the parsing stage):
my_project:
target: dev
outputs:
dev:
type: postgres
host: localhost
user: fake_user
password: fake_password
port: 5432
dbname: fake_db
schema: my_schema
threads: 1Add a file models/sources.yml with some tags on a dummy source, source table, and column test:
version: 2
sources:
- name: my_src
config:
tags: [source_config_tag] # This tag should propagate to child tests, but does not
tags: [source_legacy_tag] # This tag _does_ propagate to child tests
schema: my_schema
tables:
- name: my_tbl
config:
tags: [table_config_tag] # This tag should propagate to child tests, but does not
tags: [table_legacy_tag] # This tag _does_ propagate to child tests
columns:
- name: id
data_tests:
- unique
- not_nullRun some commands to check inheritance. Note that we have to use empty indirect selection, since eager, cautious, or buildable indirect selection will include the tests in the selection due to their relationship to the parent:
# This command should print the test IDs, but does not
uv run dbt -q ls -s "tag:source_config_tag" --indirect-selection=empty
# This command should also print the test IDs, but does not
uv run dbt -q ls -s "tag:table_config_tag" --indirect-selection=empty
# This command correctly prints test IDs
uv run dbt -q ls -s "tag:source_legacy_tag" --indirect-selection=empty
# This command also correctly prints test IDs
uv run dbt -q ls -s "tag:table_legacy_tag" --indirect-selection=emptyYou can also confirm the bug by inspecting target/manifest.json after calling dbt parse. The tags array on
source.my_project.my_src.my_tbl includes source_config_tag/table_config_tag, but the tags array on
the tests does not.
#!/usr/bin/env bash
set -euo pipefail
REPREX_DIR="$(mktemp -d)/dbt-source-tags-reprex"
mkdir -p "$REPREX_DIR/models"
cd "$REPREX_DIR"
cat > pyproject.toml <<'EOF'
[project]
name = "dbt-source-tags-reprex"
version = "0.0.0"
requires-python = ">=3.11,<3.13"
dependencies = [
"dbt-core==1.12.4",
"dbt-postgres==1.11.*",
]
# Not a distributable package, just a pinned env for the reprex.
[tool.uv]
package = false
EOF
cat > profiles.yml <<'EOF'
my_project:
target: dev
outputs:
dev:
type: postgres
host: localhost
user: fake_user
password: fake_password
port: 5432
dbname: fake_db
schema: my_schema
threads: 1
EOF
cat > dbt_project.yml <<'EOF'
name: my_project
version: "1.0.0"
config-version: 2
profile: my_project
model-paths: ["models"]
EOF
cat > models/sources.yml <<'EOF'
version: 2
sources:
- name: my_src
config:
tags: [source_config_tag] # expected to propagate to child tests, does not
tags: [source_legacy_tag] # this one DOES propagate to child tests
schema: my_schema
tables:
- name: my_tbl
config:
tags: [table_config_tag] # expected to propagate, does not
tags: [table_legacy_tag] # this one DOES propagate
columns:
- name: id
data_tests:
- unique
- not_null
EOF
echo "Reprex project created at: $REPREX_DIR"
echo
echo "Installing Python dependencies"
uv sync
echo
echo "----------------------------------------------------------------------"
echo "COMMAND: uv run dbt -q ls -s tag:source_config_tag --indirect-selection=empty"
echo "EXPECTED: source_config_tag is a config.tags value on the source, so it"
echo "should be inherited by tests, causing both tests to be listed."
echo "ACTUAL (bug): Only the source node is returned, while the tests are missing."
echo "----------------------------------------------------------------------"
uv run dbt -q ls -s "tag:source_config_tag" --indirect-selection=empty
echo
echo "----------------------------------------------------------------------"
echo "COMMAND: uv run dbt -q ls -s tag:table_config_tag --indirect-selection=empty"
echo "EXPECTED: table_config_tag is a config.tags value on the source TABLE,"
echo "so it should also be inherited by the tests, causing both tests to be listed."
echo "ACTUAL (bug): Only the source node is returned, while the tests are missing."
echo "----------------------------------------------------------------------"
uv run dbt -q ls -s "tag:table_config_tag" --indirect-selection=empty
echo
echo "----------------------------------------------------------------------"
echo "COMMAND: uv run dbt -q ls -s tag:source_legacy_tag --indirect-selection=empty"
echo "EXPECTED: source_legacy_tag is the legacy top-level 'tags' key on the"
echo "source, which should be inherited by the tests, causing both tests to be listed."
echo "ACTUAL (correct): Both test nodes are returned as expected."
echo "----------------------------------------------------------------------"
uv run dbt -q ls -s "tag:source_legacy_tag" --indirect-selection=empty
echo
echo "----------------------------------------------------------------------"
echo "COMMAND: uv run dbt -q ls -s tag:table_legacy_tag --indirect-selection=empty"
echo "EXPECTED: table_legacy_tag is the legacy top-level 'tags' key on the"
echo "source TABLE, which should be inherited by the tests, causing both tests to be listed."
echo "ACTUAL (correct): Both test nodes are returned as expected."
echo "----------------------------------------------------------------------"
uv run dbt -q ls -s "tag:table_legacy_tag" --indirect-selection=empty
echo
echo "----------------------------------------------------------------------"
echo "Ground truth: Inspecting target/manifest.json directly, independent of"
echo "node selection behavior."
echo "EXPECTED: All 4 tags should be on defined the test nodes."
echo "ACTUAL (bug): Test nodes only include the 2 legacy tags."
echo "----------------------------------------------------------------------"
uv run python3 - <<'PYEOF'
import json
m = json.load(open("target/manifest.json"))
for uid, node in m["nodes"].items():
if node["resource_type"] == "test":
print(f"node: {uid} | tags: {node['tags']}")
for uid, node in m["sources"].items():
print(f"node: {uid} | tags: {node['tags']}")
PYEOFRelevant log output
$ uv run dbt -q ls -s tag:source_config_tag --indirect-selection=empty
source:my_project.my_src.my_tbl
$ uv run dbt -q ls -s tag:table_config_tag --indirect-selection=empty
source:my_project.my_src.my_tbl
$ uv run dbt -q ls -s tag:source_legacy_tag --indirect-selection=empty
source:my_project.my_src.my_tbl
my_project.source_not_null_my_src_my_tbl_id
my_project.source_unique_my_src_my_tbl_id
$ uv run dbt -q ls -s tag:table_legacy_tag --indirect-selection=empty
source:my_project.my_src.my_tbl
my_project.source_not_null_my_src_my_tbl_id
my_project.source_unique_my_src_my_tbl_idEnvironment
- OS: Ubuntu 24.04
- Python: 3.12.13
- dbt: 1.12.4 (I also think this affects v1.10 and v1.11, see additional context below)Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context
This bug seems related to #11984, which covered the same type of bug at the column level. That bug was fixed by #11992, but that PR didn't address the issue at the source- or table-level.
I believe this bug is adapter-agnostic since it occurs at the parsing stage.
I think this bug affects all of the v1.10, v1.11, and v1.12 version lines. I haven't checked v2 because my team doesn't use it yet due to the lack of Athena support.
Related docs: https://docs.getdbt.com/reference/resource-configs/tags?version=2#backwards-compatibility-for-sources-and-exposures
Root cause
Take this root cause analysis with a grain of salt. It seems right to me, but I don't know the source code very well yet. All code snippets are permalinked to the latest commit on the v.12 version line, but you can toggle the branch to confirm that the same code applies to v.10 and v.11.
In core/dbt/parser/sources.py, SourcePatcher.parse_source_test() builds the test's tags list only from the legacy top-level attributes, ignoring target.source.config.get("tags") and target.table.config.get("tags"):
By contrast, the SourceDefinition node itself gets its tags from SourcePatcher.calculate_tags_from_raw_target(), which correctly merges legacy top-level tags and config.tags for both the source and the table:
AI disclosure
I used an LLM to help me investigate the root cause of this issue, write a first draft of this bug report, and create a reprex. I heavily copyedited the resulting text for clarity and accuracy, and I verified that all commands in the reprex run correctly on my machine and demonstrate the bug.
Source: dbt-labs/dbt-core