[v2 Bug] Snapshot ADD COLUMN emits BigQuery API type names (RECORD) instead of standard SQL types
Summary
When a snapshot's source gains a STRUCT column that the existing snapshot table does not have, Fusion generates DDL containing RECORD — the BigQuery API type name — instead of the standard SQL type STRUCT<...>. BigQuery rejects it.
ALTER TABLE `my-project`.`my_dataset`.`group_users`
ADD COLUMN datastream_metadata RECORDType not found: RECORD at [3:44]The column's real type is STRUCT<uuid STRING, source_timestamp INT64>.
Affected path
Only the snapshot column-addition path is affected:
snapshot materialization
→ adapter.get_missing_columns(staging_table, target_relation)
→ create_columns() (dbt-adapters/macros/materializations/snapshots/snapshot.sql)
→ bigquery__create_columns() (dbt-bigquery/macros/materializations/snapshot.sql)
{{ adapter.alter_table_add_columns(relation, columns) }}
→ Rust: alter_table_add_columns() (crates/dbt-adapter/src/adapter/adapter_impl.rs)alter_table_add_columns builds the clause with col.dtype():
let add_columns: Vec<String> = columns
.iter()
.map(|col| format!("ADD COLUMN {} {}", col.name(), &col.dtype()))
.collect();dtype() returns core_dtype. For BigQuery, the type construction in crates/dbt-adapter/src/column/types.rs yields:
| SqlType | dtype() |
data_type() |
|---|---|---|
Struct |
RECORD |
STRUCT<...> |
Array(Struct) |
RECORD |
ARRAY<STRUCT<...>> |
Array(other) |
inner type only, e.g. INT64 |
ARRAY<INT64> |
The correct accessor data_type() already exists on the same type; this path just calls the wrong one.
Other paths are unaffected:
bigquery__alter_relation_add_columns(used by incrementalon_schema_change) is Jinja and uses{{ column.data_type }}— correct.CREATE TABLE AS SELECTnever writes a type name.- dbt Core does not hit this at all: its snapshot column-addition goes through
SchemaField+client.update_table()rather than DDL. Core log for the same scenario:BigQuery adapter: Adding columns ([<BigQueryColumn datastream_metadata (STRUCT<`uuid` STRING, `source_timestamp` INT64>, NULLABLE)>]) to table ...
Reproduction
Same shape as #15618, with the added column being a STRUCT:
- Create a source table without the struct column and run a snapshot against it.
- Add a
STRUCT<...>column to the source table. - Run the snapshot again.
Expected: the column is added as STRUCT<...>.
Actual: Type not found: RECORD.
Additional impact: ARRAY columns fail silently
For a non-struct array such as ARRAY<INT64>, dtype() returns INT64 — the ARRAY wrapper is dropped. That does not raise an error; the column is silently added as a scalar INT64. This is arguably worse than the STRUCT case, which at least fails loudly.
Not a duplicate of
- #15618 — same function, but that was a missing database qualifier; fixed in 2.0.0-beta.1 (the DDL above is fully qualified, so that fix is present).
- dbt-labs/dbt-adapters#1702 / #1781 — a dbt Core macro issue (
get_column_schema_from_queryflattening RECORD, breakinghard_deletes: new_recordwith aUNION ALLcolumn-count mismatch); fixed 2026-07. - dbt-labs/dbt-adapters#598 — BigQuery's restriction on mutating nested record fields; a different, still-open limitation.
Versions
- Reproduced on
dbt2.0.0rc218 (preview.218), BigQuery adapter. - Still present in the source of
v2.0.4(latest release at time of writing) andmain; no related entry in CHANGELOG.
Source: dbt-labs/dbt-core